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Abstract 

New universal coding/decoding scheme for Random Access with collision detection is given in case of two senders. 
The result is used to give an achievable source-channel coding error exponent for Multiple-Access channels in case 
of independent sources. 

1 Introduction 

This paper addresses a version of the random access model of Luo, Epremides [6 and Wang, Luo [8^, which is similar 
to the model studied for one-way channels by Csiszar [2j. In the terminology of this paper, in [2] the performance of a 
codebook library consisting of several constant composition codebooks with pre-determined rates has been analyzed. 
That result shows that it is possible to achieve universally the same error exponent for each codebook as the random- 
coding error exponent of this codebook alone. This theorem is used in [2_ to give an achievable error exponent for 
joint source-channel coding (JSCC). 

This paper generalized the mentioned results of [2^ to (discrete memoryless) multiple-access channels (MACs). A 
two-senders random-access model is introduced, in which the senders have codebook libraries with constant composition 
codebooks for multiple rate choices. The error exponent of Liu and Hughes ^ for an individual codebook pair is shown 
to be universally achievable for each codebook pair in the codebook libraries, supplemented with collision detection 
in the sense of [SJ |S] . In particular, a positive answer is given to the question in jHI whether the results there are 
still valid if the receiver does not know the channel. Moreover, an achievable JSCC error exponent for transmitting 
independent sources over a MAC is given, better than that achievable by separate source and channel coding. 

Nazari, Anastasopulos, and Pradhan in [7| derive achievable error exponents for MAC's using a-decoding rule 
introduced for one-way channels in [3] by Csiszar and Korner. In the present paper a particular a-decoder is used. 
However, as the proofs follow closely [7], it can be seen that other choices could also be appropriate depending on 
actual assumptions on the analyzed models. 

Note that, an other multiterminal generalization of the JSCC result in |2j appears in Zhong, Alajaji, Campbell |9j. 
We also mention that, this paper as [2], also has connections with the topic of unequal protection of messages, see for 
example Borade, Nakiboglu, Zheng [1]. 

2 Preliminaries, notation 

Denote the set {1,2, . . . , M} by [M]. The notation follows |1] and [7] whenever possible, for example, the following 
notations are used: ViX x y), V{X\U), V^{X), T^, rp^|^(u). Let V{X\Pu) be the collection of all conditional 
distribution Vx\u for wich there exists an x G Ty^ for some u e Tp^. 

Denote Hy(X, F), Hy(X, F, U). lv{X A Y) etc. the entropy and mutual information when the random variables 
X, Y, U have joint distribution Vxy, Vxyu etc. Denote I(x A y), H(x, y) etc. the information quantities lv{X A Y), 
Hy(X, y) etc. with Vxy equal to the type of (x, y). If Vxyzu is a multivariate distribution on X x y x Z x U 
then VxYU, Vxu, Vyu etc. denote the marginal distributions respectively. Moreover, we define the following multi- 
information (see [1] exercise 3.9. and [5], but different from Yeung's notation): 

i(Xi ^X2^■■■ hXN\Y)^ H(Xi|r) + H(X2|r) + . . . 

+ H(XAr|y)-H(Xi,X2,...,Xjv|r) (1) 
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of Stochastic, Budapest University of Technology and Economics, e-mail: koitomi@math.bme.hu. The work of Tamas Koi was supported 
by the project "Talent care and cultivation in the scientific workshops of BME" (TAMOP - 4.2.2.B-10/1-2010-0009). 
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Given a MAC W : X xy ^ Z,the pentagon 

i?2) : < i?i < I(X A Z\Y, U), 
< i?2 < A Z\X, U),Ri +R2< 1{X, Y A Z\U) 

where U,X,Y,Z have joint distribution equal to PuPx\uPy\u^ i is denoted by C\W,Pij,Px\UtPy\u]- The union of 
these pentagons, i.e., the capacity region of the MAC W, is denoted by C{W). 

3 Random Access with collision detection 

In this model two transmitters try to communicate over a MAC W with one common receiver. The channel W is 
unknown to the senders and may also be unknown to the receiver (but see Remark |2| . Both senders have multiple 
codebooks with block length n. We assume that the codewords' conditional tpye on an auxiliary sequence u is fixed 
in all codebooks, however this common type can vary from codebook to codebook. 

Definition 1. Let a finite set hi, a sequence u G Z^" of type Pjj £ V"(U), positive integers Mi and M2, conditional 
distributions G V'^'iX\Pu),i G [Mi]}, {P^,^ G 7'"(3^|Pc/), j G [M2]}, rates {R\,i G [Mi]} and {P^,j G [M2]} 

he given parameters. A constant composition codebook library pair of length n with the above parameters is a pair 
{A,B) where A and B consist of constant composition codebooks (A^, . . . , A*^^) resp. {B\.. . sjjc/i that A' = 

{xl , x| , . . . xi^, } and = {y{ ,yi, . . . } with x*, G T;\ (u) and yl e T" (u) , i G [Mi] , j G [M2] , Nl 



aG [iV{], 6g [N^]. 



Before sending messages, each transmitter choses one of its codebooks independently from the other sender. Denote 
this selection by («, j) G [Mi] x [M2]. The transmitters do not share the result of their selections with each other, 
neither with the receiver. The senders send codewords x^, x^. The decoder output m is either a quadruple {i,a,j,b) 
or "collision". The receiver is required to decode quadruple (i, a,j, b) if the rate pair {R\, R2) of the choosen codebooks 
is in the interior of C[W, Pjj, Px'\Ui Py^u] ^^id to declare "collision" otherwise. Hence, two types of error are defined. 

Definition 2. For the codebooks {A'',B^), the average decoding error probability is 

Errd{i,j) = — -. — ^ Pr{m ^ m|m is sent}, 



and the average collision declaration error probability is 

ErrdiJ) 



X^meA'xBJ Pr{m 7^ "collision" |m is sent} 



To state our basic theorem we need the following notions from jS]; the index HL refers to the authors of [S]. 

Vhl=^Vhl{W,Pu,Px\u,Py\u) 

= {VuxYZ ■■ Vux = PuPx\u, Vuy = PuPy\u] (2) 
EXhl{Ri.R2.W,Pu.Px\u.Py\u) 

^ min [D{Vz\xyu\\W[Vxyu) 

yUXY Z^^HL 

+ Iv{X AY[U) + [IviX AYZ[U)~ Ri[+] (3) 
EYhl{Ri,R2,W,Pu,Px\u,Py\u) 

^ min [D{VzixYu\\W[VxYu) 

VuxY z^Vhl 

+ Iv{X AY[U) + [Iv{Y AXZ[U)-R2[+] (4) 
EXYhl{Ri,R2,W, Pu, Px\u,Py\u) 

^ min [D{Vz\xYu\\W[VxYu) (5) 

VuxY z^Vhl 

+ Iv{X A Y[U) + [Iv{X AY A Z[U) - Ri - P2I+] 
Ehl{RuR2,W,Pu,Px\u,Py\u) 

^ mm{EXHL,EYHL,EXYHL} (6) 

Theorem [1] shows that the error exponent of [5^ for an individual codebook pair is achievable for this general setting. 
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Theorem 1. For each n let constant composition random access codehook library parameters as in Definition [7] 
he given with a common set U and with ^logMi — > 0, ^logAl2 —> as ri —> oo. Then there exist a sequence 
6n{\U\, \X\, \y\, \Z\, {Mi}^^i, {Af2}^i) and for each n a constant composition codebook-library pair (A, B) with 
the given parameters, and decoder mappings with the following properties: 

(i) For all {i,j) € [Mi] x [M2] 

Errd{i,j) < ^'''^EHLiRV.Ri.W^Pu ,Px\u^Pl-\u)-^^) ^ (7) 
(ii) If {R\, R2) is not in the interior of C[W, Pu , P^^jj, Py^jj] then 

Err,{i.j) <5n- (8) 
Remark 1. Note that in part (i) of Theorem [l] i?^, W^, Pj/, Pj^l^, P^l^) > iff {R\,R{) is in the interior of 

C[W,Pu,PI,^u^Py\u]- 

The next packing lemma is an extension of Lemma 4 in [7 for this multiple codebooks setting, it provides the 
appropriate codebook library pair for Theorem [T] Note that multi information is used instead of the original notation 

of E- 

Lemma 1. Let a sequence of constant composition random access codehook lihrary parameters he given as in Theo- 
rem^ Then there exist a sequence dj^{\U\,\X\,\y\,{Mi}'^^i, {M2}'^^i) — > and for each n a constant composition 
codehook-lihrary pair {A,B) with the given parameters such that for any {i,k) G [Afi]^ and {j,l) G [M2]^ and for all 
^uxxYYU e X A- X A- X :y X 

r<"^-^W 1 <r o-«(iv(XAXAy|(7)-i?i-fl^-K^-<5^ 
"■kj\-''uxxY\ — ^ 

Ti-^'iW 1 <- o-'^i'^v{XAYAY\U)-Rl-Ri-Rl,-5'j 

^kdl^UXYYi - ^ 

U^-.Jnr 1 ^ n-n{\v{Xr\Xr<Y hY\U)-R\-R!'^-R\-R}..-&') 

^k,l l^UXXYYl — ^ ' 



where 



K]mJXY] ^ $:$:ir.,,^(u,x^y^J (9) 

a=l 6=1 



UXXY 

a=l 6=1 <:=i 

c^a it i = k 



^IfJWrl^EE Y.^r..M,<,yUa) (11) 



U XYY 

d^b if 3=1 



mVuXXYY]= (12) 

Ni Ni N^ N^ 

-EE E E lr._,(u,<,x^y^„y^). 



UXYY 

a=l 6=1 c=l d=l 

c^a if i = k dj^b if j = L 



Sketch of proof: Let {A,B) be a random constant composition codebook library pair, i. e for all i G [Afi], j G [M2] the 
codewords of A* , are ch 

following random variable: 



codewords of A* , are chosen independently and uniformly from Tpi (u) and T"j (u) respectively. Let T be the 



Ml M2 Ml M2 

^^-EEEE E k'jiw] 



i=lj = lfe=l/=l ^uxxyy"^ 

V^{Uy.XxXxyy.y) 



^ 2n(lv(XAY\U)-R'-^^-R{-S'^) 

+ K^'^\V ■ ]2^(iv{XAXAY\U)-R{-Ri~R'i~s'j 

4- K^'^W .]r>HiviXAYAY\U)-Rl-R.i-Rl,-5',J 

-I- K^'iW ]r,n{lv{XAXAYAY\U)-R{-Rl,-R'i) 
^k,iruXXYY\'^ 

.2"(-«2-5l)l. (13) 
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Using basic properties of types (similarly as in [51 17]) it can be seen that we can choose (5„ — > such that E[T] < 1. 
Hence there exists a realization of the codebook library pair with T < 1. Taking into account the positivity of the 
terms of T, the lemma is proved. ■ 

Sketch of proof of Theorem Q]' Lemma ^ provides the appropriate constant composition codebook-library pair 
{A,B). To construct the decoder, define a : P(W xA'x3^xZ)^i?by a{VuxYz) = IviX AY A Z\U). In the 
first stage of decoding, the receiver determine the indices k e [Mi], I e [M2], a G [N^], b G [iVj] which maximize 
a(u, x^,y^,z) — Ri — R2, where z denotes the output sequence and a is evaluated on the joint type of (u,x^,y^,z). 
In the second stage, to deal with collisions, the decoder checks the following three inequalities: 

/(x|Ay[Az|u)-i?f-4 >r;„ (14) 
/(x|Ay^,z|u)-i?f >,?„ (15) 
/(yf a4,z|u)-4 >r,„, (16) 

where r]n{\U\, \X\, \y\, \Z\,{Mi}^^i, {M2}^i) — >■ is an appropriately chosen positive sequence. If the above three 
inequalities are fuUfiUed then the decoder decodes x|, y^ as the codewords sent, if at least one of them is not fuUfiUed, 
then the decoder reports "collision". 

Some details about the necessary calculations can be found in the Appendix. ■ 

Remark 2. Other a-decoders can be also used; if the receiver knows the channel W, the a function can depend on 
W. Note also that for the sake of brevity the expurgation method for multiple-access channel in [7 is not used in this 
paper. However, it is possible to prove an expurgated version of Lemma[T]which leads larger achievable error exponent 
for small rates. 



4 Source-channel coding 

Let two independent discrete memoryless sources (DMS) Qi,Q2 with alphabets Si,S2 be given. We want to transmit 
these sources over MAC W. It is assumed that the sources Qi and Q2 and the channel W are known by the encoders, 
however, not known by the decoder. 

Definition 3. A source-channel code of length n is a mapping triple (/i,/2,¥') with encoders fi : 5" — > X" , /2 : 
3^" and decoder v? : Z" ^ 5f x 



Definition 4. The error of a source-channel code (/i,/2,V') of length n is defined by 

Err{fuf2,^)= Q?(sl)Q'2'(s2)Pe(Sl,S2), 



(17) 



(si ,s2)e 



whe 



Pe(si,S2) = W^{{Z G Z" : ^(Z) ^ (S1,S2)}|/1(S1),/2(S2)). 



We assume that {H{Qi), H{Q2)) is in the interior of C{W). In this case Qi,Q2 can be reliable transmitted over 
channel W, for example, by separate source and channel coding. Regarding error exponents, by separate coding the 
exponent Eshl{Qi,Q2,W) is achievable, where 



Eshl{QuQ2,W) 



max mm 

0<Hi <log\Si 
a<R2<loglS2l 



ei(i?i,Qi), 



e2iR2,Q2),EHL{Rl,R2,W) 

ei(i?i, Qi),e2(i?2, Q2) are the source reliability functions 



and 



e.(i?«,g.)= min DiP\\Q,), ^€{1,2}, 

P:H{P}>Ri 



Ehl{Ri,R2,W)^ 

= sup sup Ehl[Ri,R2,W,Pu,Px\u,Py\u]- 



(18) 



(19) 



(20) 



Pc/e^<") p- 



Note that EsHLiQi,Q2iW) > 0. Of course, (18 1 could be improved replacing E^LiRii R2^W) by the reliability 



function of channel W which, however, is not known in general. 
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In this section a larger exponent then Eshl is given using JSCC. 

For arbitrary U and Pij G ViU) let GiiU) and G2{U) be the set of all continous mappings [0, ZogjS'il 
and [0, ^0(7|iS'2|] — > V{y\U) respectively, and define 



V{X\U) 



Ej{Q^,Q2,W)^ SMv sup Ej{Q^,Q2,W,Pu. 91,92) 

U gieGi(M) 



where 



E]{Qi,Q2,W,Pu, 91,92) 



mm 

0<Ri <log\Si 
0<R2<loglS2l 



ei{Ri) + e2{R2) 



+ EHL{Ri,R2,W,Pu,gi{Ri),92{R2)) 



(21) 



Proposition 2. Eshl{Qi,Q2,W) < Ej{Qi,Q2,W). 



Proof: Restricting the suprerum to constant functions 91, 92 in (21 1, we see that: 

Ej{Qi,Q2,W)> sup sup min ei(i?i) 

" PvlTiEV(X\U} 0<Ri<log\Sil _ 

|M) 

+ e2{R2) + Ehl{Ri,R2,W, Pu,Px\u,Py\u)]- (22) 

Using the definition of ei(i?i), e2(i?2) and Ehl{Ri, R2,W, Pjj , Px\Ui Py\u)] it can be easily seen that the right-hand 
side of (22 1 is greater than or equal to Eshl{Qi, Q2, W). ■ 

Remark 3. Actually, the strict inequality holds, except in very special cases. 

The following theorem shows that Ej{Qi,Q2,W) is an achievable error exponent for this source-channel coding 
scenario, hence JSCC leads to larger exponent than separate source and channel coding. More exactly, we show that 
for any choice of Pu, 91, 92, the exponent Ej{Qi,Q2,W, Pu,9i, 92) is achievable even if the senders do not know the 
sources and the channel; if they do know them, they can optimize in Pjj, gi, 92, to achieve Ej{Qi, Q2, W). 

Theorem 3. Let lA, Pjj € VilA), 91 G Gi{U) and 92 G G2{U) be given. There exist a source- channel code for each n 
and a sequence Vn{\Si\, \S2\, \y\, \Z\) — ^ with 

Err{fij2,(p) < 2-"(-^jWi''32.W'.^'if.9i>ff2)-!^.), (23) 

Sketch of proof: Approximate uniformly Pu, 91, 92 by sequences Puin] £ V'^{U), 9i[n] : [Oj/oglS*!!] V{X\Pu{n)), 
g2[n]:[0,log\S2\]^V{y\Pu{n)). 

Let u e Tp^[r{\ be arbitrary sequence. Choose Mi = |'P"(5i)| and M2 = |7'"(52)|. Let P^ , P^ , . . . , P^^' and 
P^,Pi,...,P^^^ denote all possible types from T"'{Si) and ^{82) respectively. For all i € [Mi], j e [M2] let Nl 

and N2 be equal to \Tpi \ and \Tpj \ respectively, and let Px\u and Pyuj be equal to gi[n]{Nl), (72 ["«.]( -/V2) respectively. 

01 -'2 ' ' 

with these parameters consider the resulting codebook library pair (A, B) and the decoder 

mapping cj) satisfying ([t]) for all (i, j) e [Mi] x [M2]. 

Let /i : 5" — ?> X" and /2 : 5^ — > 3^" be the mappings which map each Tpi and T"j to and respectively. Let 

1 ^2 

tf : Z" — > Si X S2 be the mapping which first determines a codeword pair from {A, B) using 0, then uses the inverse 
of fi and /2 to determine the source sequences. The crucial step is the following equation 



Ml M2 

i?rr(/i, /2, ^) = ^ 5] gr('r^0Q2 (T^. ) 

1=1 j=l 



Note that the second line of (24) is Errd{i,j) in the terminology of Theorem [l] Hence substituting ([7| into (24) and 
using (19) and standard proprties of types, this theorem is proved. ■ 

Remark 4- Analogously to Lemma 2 of [2], it can be shown that the error exponent can not be greater than 

min [ei{Ri) + e2{R2)+E{Ri,R2,W)] 

0<Ri<log\Si I 
0<H2<'os|S2 

where E{Ri, R2, W)] is the reliability function of channel W. 
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[Sketch of proof of Theorem [l] Let us define the following sets for all i E [Mi], j e [A/2], a e [N'l], b E [iV^] 



^a.b — 



> a(u, xjf,y^,z) - i?J - i?2,for all 

k E [Ml], I E [M2], c E [N^],d E [N^] 



(25) 



Qi J A 



Then for all E [Mi] x [M2]: 



z : A yl A z|u) - R\ - R^ < 77,, or ' 
KK A Yb, z|u) - i?l < ?]n or 
I(y>x;,z|u) -i?^2 <?7« 



(26) 



"1 "i 
1 2 a— 1 6—1 



+ 



1 



n; Ni 



i J a—l b—1 

Ni Ni 



d=l 
djtb 

N' Ni 



c=l d=l 
' N^ 



Ml A-i Ni N~ 

E^EE^" U<.^K>y^. 

Mo . Ni Ni / Ni, 



E^EE^"(U<.i<> 

= 1 ^^1^^2 Q=l b=l 



yi 



Ml M2 NI N^ ( Ni Ni, 

EE^EE^" UU<^^K> 

fc=l ( = 1 ^V^^2 a=16=l \c=ld=l 



yl 



(27) 



For the sake of brevity, we introduce the following notations for the terms of the right-hand side of equation (27 1: 

Errd{i,j) < th'''-' + error X*'^ + errorY^'^ 



Ml 



+ error XY^'j + error X^f^ 



k=l 



M2 



Ml M2 



errorY^ f + errorXY^*'j 



(28) 



i=i 

Ma 



fc = l 2 = 1 



Let us define the following expressions for all i E [Mi], j E [M2], k E [Mi], I E [M2]: 



A j ^(VUXYZ) - R\ < «( Wz) - 



(29) 



{^UXYYZ '■ 
a{VuxYz) - Ri < a{Vi 
Vux - PuP'xw^^UY = PuPi.\u^ 
^UY ^ PuPy\u- 



UXYz) 



(30) 
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k.l 



^UXYXYZ • 

a{VuxYz) - R\ 



Vux 

VuY 



PIJPI 



PuP; 



Y\U' 



)-R'[ 
V, 



ux 



^2: 

= PuP^ 



UY 



PuP. 



Y\U- 



(31) 



min ^ D{Vz\xYu\\W\VxYu) 

^UXY X Z^^^ k.l 



+ lv{X A Y\U) + I lv{X A XrZ|[/) - R{\ 



» 1 + 



min D{^z\XYU \\W\VXYU) 

+ iv{x A y|?7) + 1 iv{Y A xyz|c/) - i?^2l+ 

EXY^j^ min ^ ^. i?(l^z|xyc/| VxYC/) 

+ Iv(XAy|t/)+ IviXY AXYZ\U) 



-lv{X AY\U) -R\~Ri 



(32) 



(33) 



(34) 



Relating the error probailities to packing functions ([9|-p2|) as in [7 gives: 

errorXij^i < 2-"(^^^"-^"\ errori^f , < 2-"(^^"-^") 



error XY^j < 2-"^^^^^-^-) 



for some sequence (5„(|iY|, \y\, \Z\, Mi, M2) 0. Moreover, using definitions (^-(|6| and (34l we get: 



EX-j > EX^,, EYlj > EYlii 
EXY^^ > EXYlii. 



Furthermore, using standard properties of types it follows that th^^'^^ < 2 where ETH{i,j) is defined 

by 



where 



min D{Vz\xYu\\W\VxYu) + lv{X AY\U) 



VuXYZ ■■ Vux = PuP\\u^ ^VY = PuP'y\v 
\v(X hY,Z\\J) - R\ < r/„ or 
lv{Y AX,Z\U) - Ri < r]n or 
Iy(X A r A Z\U) - R\ -R{<rin 



(35) 



Using that \mn{EX^^j^, EYj^'f , EY^fj^) < ETH{i, j) + ?7„, the above inequalities prove part (i) of Theorem jlj 
To prove part (ii) the following bound is useful: 



ErrS,j) < T^EE^"(^ ^ ^fc G [A'h],3l G [Phi 



3c G [Nfl3d e m such that z ^ 0';iK,yi 



(36) 



Using union bound, it is possible to expand (36 1 same way as Errc{i,j) is expanded in (27). Then the term corre- 
sponding to case {k,c,l,d) — {i,a,j,b) can be upper bounded using standard properties of types. This upper bound 
leads to exponent mmy^^^^f^Qi.j D{Vz\xyu\\^\Vxyu) + Iv{X AY\U). The other cases can be upper bounded using 
the technique of ^ by exponent similar to (32 1- (34), the sets on which the minimum is taken are different. Using the 
properties of these sets all terms within the positive part sign | . . . |^ can be lower bounded by rjn. Altogether, it can 
be seen that Errc{i,j) is small, if rjn goes to appropriately slow. 
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